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Background / Context: 

These studies are especially well-suited to the conference’s theme, the role of replication. The 
six studies follow-up on positive findings from an earlier randomized controlled trial of 
perfonnance based scholarships for community college students. They use randomized 
controlled trials to evaluate the impact of performance based scholarships on community college 
students’ academic perfonnance across the United States, targeting different segments of each 
college’s student population. In each program, students had to meet clear academic benchmarks 
in order to earn the scholarship payment. Except for at one site, the programs showed positive 
impacts on the percentage of students who met the academic benchmark, and generally, the 
programs showed positive impacts on the number of credits students earned in the first year, 
although the results were less consistent for outcomes such as persistence. The present research 
explores the replication results for across the six sites in greater detail, and also considers the 
broader implications of the studies when considered together. 

Community colleges - and postsecondary education more generally - have come to the forefront 
of civic dialogue in an economy that compels the public sector to do increasingly more with 
fewer resources. Policymakers, education leaders, and communities across the country recognize 
the need to improve college attendance and success in order to meet the increasing demand for a 
more educated workforce, but are faced with the realities of the current budgetary environment. 
At the same time, students are facing mounting costs to attend college, and financial aid has not 
been able to keep pace with these costs. This affects both college enrollment and persistence in 
college, and low-income students are especially disadvantaged. Evidence suggests that financial 
aid is positively associated with enrollment and with increased persistence (St. John, Musoba, 
Simmons, and Chung (2002), Kane (2004), Dynarski (2000, 2003), Cornwell, Mustard, and 
Sridhar (2006).). Prior to these studies, relatively little was known about the effectiveness of 
scholarships on student retention and academic perfonnance, despite the prominent role 
scholarships have played in public and private support for higher education. 

Purpose / Objective / Research Question / Focus of Study: 

Perfonnance-based scholarships were developed to help tackle some of the financial obstacles 
facing students in the postsecondary education system. In general, these scholarships aim to help 
reduce the financial burdens of low-income college students, and are structured to help 
incentivize good academic progress. The demonstration seeks to answer several questions: 

1. What is the impact of these scholarships on short-term academic outcomes, including credit 
attempts and credit completions? 

2. What is the impact of these scholarships on longer-tenn academic outcomes, including 
persistence in school and attainment of degrees and certificates? 

3. Do the scholarships work in diverse settings, and for different types of students? 

Setting: 
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Performance-based scholarships were evaluated in six states across the U.S. with eight 
educational institutions and one state-level intennediary. The postsecondary institutions involved 
were all community colleges or non-selective four-year colleges, all of whose populations 
contain predominantly low-income students. 

Population / Participants / Subjects: 

More than 12,000 low-income college students enrolled in the study and were randomly assigned 
to either a program group, which was eligible to receive a perfonnance-based scholarship, or a 
control group, which was not eligible. Some states focused on particular populations of interest, 
including Latino male students, students in need of developmental (remedial) math courses, and 
students who are parents (please see Table 1 for more details about the student populations). 

Intervention / Program / Practice: 

The scholarship program varies a bit from state to state, but the basic structure is the 
same in all states. Students are generally paid at multiple points during the semester if they earn a 
certain number of credits with a “C” or better grade. By conditioning additional financial aid on 
meeting certain perfonnance benchmarks, the programs seek to encourage students to focus on 
their studies, which, in turn, should lead them to perform better in their classes in the short tenn. 
In the medium tenn, they should progress through their degree requirements more quickly, 
which may then lead to long-term gains such as graduation or transfer to a four-year college. 
Finally, if the scholarship promotes academic success, the intervention could lead to improved 
labor market outcomes, including higher earnings. 

Importantly, the scholarships are paid directly to students, allowing them to use the funds 
for their most pressing needs (whether books, child care, or other financial obligations that might 
disrupt their studies). Students have complete discretion over how they use the funds, although 
most report using it to help with a range of essential expenses. The scholarships are paid in 
addition to Pell Grants — the main federal source of need-based aid — and other existing financial 
aid programs, including state and institutional grants. In this way, students have more funds to 
cover academic and living expenses and can potentially reduce their dependency on loans. 

Lastly, unlike merit-based aid, performance-based scholarships are paid to students based on 
their academic performance in the current tenn, regardless of what happened in previous terms. 
That is, eligibility for the scholarship is not based on prior performance like high school GPA. 

Students were offered the scholarship for between two and four semesters and were 
eligible to earn between $500 and $1,500 per semester. The duration and value of the 
scholarships varied by state (please see Table 2 for a summary of the performance-based 
scholarship interventions at each site). 

Research Design: 

The studies each use a random assignment research design, the “gold standard” in program 
evaluation. Random assignment creates two groups of students that are similar in characteristics 
that can be measured, such as age and gender, and in those that are more difficult to measure, 
such as tenacity and motivation. As a result, subsequent differences in outcomes (for example, 
credit accumulation) can be attributed with a high level of confidence to the program. 

Data Collection and Analysis: 
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The study gathered data from a number of sources to track students’ academic outcomes. 
In most states, academic measures were created from transcript data provided by the colleges 
themselves, and are based on outcomes at the college of random assignment only. In one state, 
academic measures were created from data provided by the National Student Clearinghouse and 
are based on data from all colleges that report to the Clearinghouse - more than 96% of 
American colleges and universities. In another state, academic measures were created from data 
provided by the state Board of Regents that covers all public and private two- and four- year 
colleges in the state. 

Findings / Results: 

In the first year after random assignment, in general, perfonnance-based scholarships: 

• Increase the proportion of students who met the academic benchmark incentivized by the 
scholarship . In all sites except New York, students in the program group met the academic 
benchmark at higher rates than students in the control group in one of the two semesters. 
These increases are sizeable. In the first tenn, they range from 7.0 percentage points in 
Ohio to 9.6 percentage points in Florida. In the second tenn, they range from 7.3 
percentage points in Ohio to 17.6 percentage points in New Mexico. 

• Increase the number of credits earned at the end of the first program year . With the 
exception of Florida and New Mexico, students who are eligible for perfonnance-based 
scholarships earn more credits by the end of the first program year, compared to students in 
the control group. The impact ranges from about one credit earned over the year in New 
York to 1.7 credits earned in Arizona and Ohio. 2 For context, a typical course at these 
colleges bears 3 or 4 credits. 

• Have mixed impacts on the proportion of students who registered for college in the second 
semester of the program . In two states, there are small impacts on second-semester 
registration (4.6 percentage points in Arizona and 3.8 percentage points in California). In 
the remainder of the sites in the demonstration, there are no statistically significant impacts 
on registration in the second semester of the program. 

In the second year after random assignment, in general, perfonnance-based scholarships: 

• Had some impact on credit accumulation as of the end of the second year . In Ohio and 
New Mexico, there is about a two credit impact on credits earned by the end of the second 
year after random assignment. Notably, this impact has grown since the end of the first 
year. In New York, there are no statistically significant impacts on credits earned by the 
end of the second year. 

• Continued to increase the proportion of students meeting the academic benchmark . In New 
Mexico, we find an 18 percentage point impact on meeting the benchmark in the third 
semester of the program and a 12 percentage point impact on meeting the benchmark in the 


2 The impact on credits earned can be thought of in terms of full-time equivalence (FTE) as well. An impact of two 
credits earned per program group member converts to 2,000 credits earned for 1,000 students, or 167 FTEs. 
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fourth semester of the program. Notably, in Ohio, we also find impacts on meeting the full- 
time academic benchmark in one of the semesters, even though the scholarship was no 

longer available during that semester (around 3.2 percentage points). 

In the site with the longest follow-up so far, there have been impacts on degree attainment. Starting in 
year two, there is a 3.6 percentage point impact on earning any degree; at the end of year three, this 
impact is largely unchanged at 3.5 percentage points. Around 23 percent of control group students have 
earned a degree or certificate three years after random assigmnent, compared to 26.9 percent of program 
group students, representing an increase of around 15 percent. This is mostly driven by an impact on 
earning an associate’s degree at the end of the third year. At some colleges, the program also lowered 
student loan debt (please see Tables 3- 6 for more details about program impacts). 

A number of pre-specified subgroups were analyzed in order to determine if these scholarships 
work better for some types of students compared with others. Overall there are statistically 
significant impacts on credits earned in the first year of the program, across all subgroups; both 
men and women; Latino students; students who are parents; traditionally-aged and older 
students; students who are first in their families to attend college; and students who were 
employed at random assignment. However, the difference between the subgroups is not 
significant (except in the case of gender). For example, parents in the program group earned 
about 1 .4 credits more than parents in the control group, over the first year of the program. 
Additionally, non-parents in the program group earned about 1 . 1 credits more than non-parents 
in the control group. However, the difference between these two impact estimates is not 
significant. This is encouraging — it means that the scholarships worked across a variety of 
different types of students (including a number of at-risk groups), and generally didn’t work 
better or worse for any particular type of student (please see Tables 7 and 8 for more details 
about subgroups analysis). 

Conclusions: 

Perfonnance-based scholarships are a promising way to tie additional financial aid to 
academic benchmarks and improve outcomes for low-income students. These studies show that 
perfonnance-based scholarship programs can be replicated in diverse contexts across the United 
States and that they work for different types of students in these settings; the scholarships 
generally have positive impacts on students’ academic perfonnance. Together these trials 
produce highly reliable estimates of the impacts of perfonnance-based scholarships. 

This research also demonstrates that large scale replications using randomized controlled 
trials are possible in community colleges, and that the results can be informative not only to the 
participating sites, but to the field more broadly. States and scholarship-granting organizations 
seeking to be more purposeful with their scholarship dollars may want to consider perfonnance- 
based scholarships as a way to incentivize academic success while providing needy students with 
additional financial aid. 
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The Performance-Based Scholarship Demonstration 
Table 1 


Selected Characteristics of Sample Members at Baseline, by Program 


Characteristic (%) 

Opening Doors 
Louisiana 

Arizona 

California 

Florida 

New 

Mexico 

New York 

Ohio 

Gender 

Male 

7.6 

100.0 

39.8 

33.7 

39.2 

30.9 

13.6 

Female 3 

92.4 

0.0 

60.2 

66.3 

60.8 

69.1 

86.4 

Average age (years) 

25.3 

24.3 

17.6 

27.0 

18.0 

26.5 

29.9 

Race/Ethnicity b 

Hispanic 

2.6 

99.7 

60.7 

30.7 

60.6 

44.3 

8.6 

White 

10.5 

0.1 

20.3 

30.5 

21.8 

6.1 

54.6 

Black 

84.9 

0.0 

3.8 

33.1 

2.7 

37.2 

31.4 

Asian or Pacific Islander 

0.4 

0.0 

11.2 

1.2 

3.5 

9.7 

0.6 

Other 

1.6 

0.2 

3.9 

4.4 

11.3 

2.7 

4.8 

Have any children 

100.0 

24.6 

NA 

44.1 

1.8 

47.8 

99.7 

Household receiving any government benefits L 

70.7 

37.5 

NA 

38.0 

NA 

37.9 

65.8 

Missing 

0.4 

12.4 

NA 

12.6 

NA 

12.0 

4.0 

Currently employed 

51.6 

44.9 

NA 

50.8 

49.0 

56.0 

48.8 

Diplomas/degrees earned 11 

High school diploma 

77.9 

83.1 

NA 

75.4 

97.8 

65.0 

74.2 

GED 

18.6 

14.7 

NA 

26.3 

1.3 

33.1 

22.7 

Occupational/technical certificate 

9.8 

7.9 

NA 

13.7 

1.0 

13.3 

15.3 

Associate's degree or higher 

0.5 

2.1 

NA 

1.3 

0.2 

2.6 

0.6 

None of the above 

3.1 

2.6 

NA 

0.1 

0.7 

2.4 

2.2 

First person in family to attend college 

42.6 

36.7 

53.7 

33.3 

32.8 

32.9 

30.0 

Sample size (total = 12,632) 

1,019 

1,028 

4,642 

1,075 

1,081 

1,502 

2,285 


(continued) 







Table 1 (continued) 


SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: NA = Not available because question was not asked on BIF for this site. 

Calculations for this table used all available data. 

Missing values are only included in variable distributions for characteristics with more than 5 percent of the sample missing. 

a Female students are not eligible to participate in the program at Pima Community College; thus, gender was imputed and not explicitly asked on the BIF. 
b Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. Respondents who said they are not Hispanic and 
chose more than one race are included in the Other category. These respondents, combined with those who said they were American Indian or Alaskan Native 
or another race/ethnicities, are included in "Other." 

c Benefits include unemployment/dislocated worker benefits, Supplemental Security Income (SSI) or disability benefits, cash assistance or welfare, food 
stamps, and Section 8 or public housing. 

d Distributions may not add to 100 percent because categories are not mutually exclusive. 



The Performance-Based Scholarship Demonstration 
Table 2 

Summary of PBS Interventions 


Opening Doors 
Louisiana 

PBS 

Arizona 

PBS 

California 

PBS 

Florida 

PBS 
New Mexico 

PBS 
New York 

PBS 

Ohio 

Sample Selection Criteria 







Age 

18-34 

No age criteria 

16-19 

At least 18 

17-20 

22-35 

At least 18 

Low-Income 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Additional 

Requirements 

Must be a parent 
Earned a HS 
diploma, GED, or 
passing score on 
college entrance 
exam 

Hispanic males 
< 45 credits earned 

Must be 1st year 
student 
Must have 
completed 
FAFSA and Cal 
Grant 

In need of 
remedial 
math 

Must be 1st 
year student 

Live away from 
parents 
In need of at 
least 1 remedial 
course 

Must be a 
parent 

Intervention 








Scholarship 

amount per term 

$1,000 

Up to $1,500 

$333 -$1000 

$600 

$1,000 

$1,300 

$300 - $900 

Scholarship 

duration 

2 semesters 

3 semesters 

1-4 semesters 

3 semesters 

4 semesters 

2 semesters plus 1 
summer semester 3 

2 semesters 

Scholarship 
disbursement 
per term 

3 

3 

Varies 

2 

3 

3 

1 

Max amount 

$2,000 

$4,500 

$4,000 

$1,800 

$4,000 

$2,600 to $3,900 a 

$1,800 

Years of program 
operation 

Spring 2004 to 
Summer 2005 

Fall 2010 to 
Fall 2012 

Fall 2009 to 
Spring 2012 

Fall 20 10 to 
Fall 2012 

Fall 2008 to 
Spring 2011 

Fall 2008 to 
Summer 2010 

Fall 2008 to 
Winter 2010 
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Table 2 (continued) 


Opening Doors 
Louisiana 

PBS 

Arizona 

PBS 

California 

.if .wtyi 

PBS 
New Mexico 

PBS 
New York 

PBS 

Ohio 

Total sample 

1,019 

1,028 

4,642 

1,076 

1,081 

1,502 

2,285 

Performance Benchmarks 







Academic 

criteria 

Enroll in college, 
and complete at 
least 6 credits 
with a "C" or 
better average 

Part time: 6-11 
credits with "C" or 
better in each 
Full time: 12 or more 
credits, "C" or 
better in each 

Enroll in college, 
and complete at 
least 6 credits 
with a "C" or 
better average 

Complete a 
sequence of 
math courses 
with a "C" or 
better 

Complete at least 
12 (first semester) 
or 15 (subsequent 
semesters) credits 
with a "C" or 
better average 

Enroll in college, 
and complete at 
least 6 credits 
with a "C" in 
each 

Part time: 6-11 
credits with "C" 
or better in each 
Full time: 12 or 
more credits, "C" 
or better in each 

Service criteria 

Meet with 
advisor 

Meet with advisor, 
complete tutoring 
and workshop 
requirements 

N/A 

Complete 

tutoring 

requirements 

Meet with advisor 

N/A 

N/A 

Participating Institutions 







Participating 

Institutions 

Delgado 

Community 

College 

Louisiana 

Technical 

College 

Pima Community 
College 

California 
Student 
Aid Commission 
and LA Chamber 
of Commerce 
operate Cash for 
College 
workshops and 
the PBS 
program. 

Flillsborough 

Community 

College 

University of New 
Mexico 

Hostos 

Community 

College 

Borough of 
Manhattan 
Community 
College 

Lorain County 
Community 
College 

Owens 

Community 

College 

Sinclair 

Community 

College 


SOURCE: Performance-Based Scholarship Study Design. 


NOTES: Scholarships are limited to the institution where the evaluation occurred, with the exception of PBS California. For this site, scholarships were portable 
to any accredited, degree-granting, 2-year, or 4-year post-secondary institution in the U.S. 

a New York had a 3-way design where half of all New York program group students are eligible to receive a summer scholarship. 









The Performance-Based Scholarship Demonstration 
Table 3 

Year One Academic Outcomes Among Sample Members 


Program 

Sample 

Size 

Program 

Group 

Control 

Group 

Difference 

Percent 

Change 

Registered in second program term (%) a 

Opening Doors Louisiana 

537 

57.1 

38.9 

Jg J H=H=H= 

46.6 

Arizona 

1,028 

78.6 

74.0 

4.6 * 

6.2 

California b 

4,642 

87.6 

83.9 

2 g *H=H= 

4.5 

Florida 

1,075 

78.6 

77.1 

1.6 

2.0 

New Mexico 

1,081 

91.6 

91.2 

0.4 

0.5 

New York 

1,502 

78.1 

76.6 

1.6 

2.0 

Ohio 0 

2,285 

73.8 

71.1 

2.7 

3.8 

Credits earned as of end of first year, post-RA 

Opening Doors Louisiana 

537 

11.0 

7.7 

3 3 *** 

43.7 

Arizona 

1,028 

16.1 

14.3 

2 7 ** 

12.2 

Florida 

1,075 

14.9 

13.9 

1.0 

7.3 

New Mexico 

1,081 

25.7 

24.8 

0.9 

3.7 

New York 

1,502 

16.3 

15.5 

0.9 * 

5.6 

Ohio 

2,285 

15.6 

13.9 

2 7 *** 

12.1 

Met academic benchmark in first program term (%) d 

Opening Doors Louisiana 

537 

43.7 

29.5 

24 2 *** 

48.2 

Arizona 

1,028 

37.9 

29.4 

8 5 *** 

28.9 

Florida 

1,075 

55.7 

46.0 

g *** 

21.0 

New Mexico 

1,081 

80.2 

78.5 

1.7 

2.2 

New York 

1,502 

71.4 

70.5 

0.9 

1.3 

Ohio 

Met academic benchmark in second program term (%)' 

2,285 

B 

33.2 

26.3 

7 Q *** 

26.4 

Opening Doors Louisiana 

537 

32.5 

17.3 

25 2 *** 

87.8 

Arizona 

1,028 

28.3 

16.9 

2 2 4 *** 

67.6 

Florida 

1,075 

29.4 

21.4 

8 o *** 

37.5 

New Mexico 

1,081 

49.8 

32.3 

27 *** 

54.4 

New York 

1,502 

57.4 

56.8 

0.6 

1.1 

Ohio 0 

2,285 

24.9 

17.6 

7 3 *** 

41.2 
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Table 3 (continued) 


SOURCES: Calculations using National Student Clearinghouse, Delgado Community College, Louisiana 
Technical College, Pima Community College, Hillsborough Community College, University of New Mexico, 
Borough of Manhattan Community College, Hostos Community College, and Ohio Board of Regents transcript 
data. 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates for all sites, excluding California, are adjusted by research cohort and campus. For California, 
estimates are adjusted by research cohort and workshop region. 

Only the first two cohorts are shown for Opening Doors Louisiana. 

Represents the second term over which program group students were eligible for a scholarship. 

Represents data pulled from the National Student Clearinghouse. Data were not found for 305 students (6.6%) 
of the sample. 

c Due to the timing of scholarship-eligible terms, this measure is calculated slightly differently for students at 
Sinclair Community College. For students in the fall 2008 cohort, this represents registration in the spring 2009 
quarter. For students in the winter 2009 cohort, this represents registration in the summer 2009 quarter (over which 
students were not eligible for a scholarship). For students in the spring 2009 cohort, this represents registration in 
the fall 2009 quarter. For students at Lorain County and Owens Community Colleges, depending on the cohort, this 
represents the fall or spring term. 

d Due to the timing of scholarship-eligible terms and data limitations, some of these measures are proxies. For 
Louisiana, this represents the proportion of students who earned 6 or more credits with a "C" or better overall in 
their first term (depending on the cohort, this could be a fall, spring or summer term). For Arizona, this represents 
the proportion of students who earned 12 or more credits with a “C” or better in their first term (depending on the 
cohort, this could be a fall or spring term). For Florida, this represents the proportion of students who earned a "C" 
or better in MAT 0024/0028 in their first term (depending on the cohort, this could be a fall or spring term). For 
New Mexico, this represents the proportion of students who earned 12 or more credits with a "C" or better overall 
in their first term (a fall term), and includes students who earned the award over the winter intersession. For New 
York, this represents the proportion of students who earned 6 or more credits with a "C" or better in their first term 
(depending on the cohort, this could be a fall or spring term). For Ohio, this represents the proportion of students 
who earned 12 or more credits in their first term (depending on the cohort, this could be a fall, winter, or spring 
term). 

e Due to the timing of scholarship-eligible terms and data limitations, some of these measures are proxies. For 
Louisiana, this represents the proportion of students who earned 6 or more credits with a "C" or better overall in 
their second term (depending on the cohort, this could be a fall, spring or summer term). For Arizona, this 
represents the proportion of students who earned 12 or more credits with a “C” or better in their second term 
(depending on the cohort, this could be a fall or spring term). For Florida, this represents the proportion of students 
who earned a "C" or better in MAT 1033 in their second term (depending on the cohort, this could be a fall or 
spring term). For New Mexico, this represents the proportion of students who earned 15 or more credits with a "C" 
or better overall in their second term (a spring term), and does not include students who earned the award over the 
summer. For New York, this represents the proportion of students who earned 6 or more credits with a "C" or better 
in their second term (depending on the cohort, this could be a fall or spring term). For Ohio, this represents the 
proportion of students who earned 12 or more credits in their second term (depending on the cohort, this could be a 
fall, winter, or spring term). 



The Performance-Based Scholarship Demonstration 
Table 4 

Year Two Academic Outcomes Among Sample Members 


Program 

Sample 

Size 

Program 

Group 

Control 

Group 

Difference 

Percent 

Change 

Registered at the beginning of second year (%) 

Opening Doors Louisiana 

537 

30.1 

22.9 

7 2** 

31.6 

California 3 

4,642 

81.3 

78.9 

2.4 * 

3.0 

New Mexico 

1,081 

77.6 

78.5 

-0.9 

-1.2 

New York 

1,502 

61.9 

60.7 

1.2 

2.0 

Ohio 

2,285 

60.5 

59.2 

1.3 

2.2 

Credits earned as of end of second year, post-RA 

Opening Doors Louisiana 

537 

13.7 

10.0 

3 y *** 

36.9 

New Mexico 

1,081 

46.5 

44.3 

2.2 * 

5.1 

New York 

1,502 

26.7 

25.6 

1.1 

4.3 

Ohio 

2,285 

24.8 

22.4 

2 4 *H=H= 

10.6 

Met academic benchmark in third term (%) b 

Opening Doors Louisiana 

537 

14.7 

11.1 

3.6 

32.3 

New Mexico 

1,081 

50.7 

32.3 

Jg 4 H=H=H= 

57.1 

New York 

1,502 

44.0 

44.2 

-0.2 

-0.4 

Ohio 

2,285 

16.1 

13.8 

2.3 

16.5 

Met academic benchmark in fourth term (%) c 

Opening Doors Louisiana 

537 

4.9 

5.2 

-0.3 

-5.2 

New Mexico 

1,081 

42.4 

30.3 

Y 2 j *** 

39.9 

New York 

1,502 

33.4 

34.1 

-0.7 

-1.9 

Ohio d 

2,285 

12.5 

9.4 

3 2** 

33.8 


(continued) 







Table 4 (continued) 


SOURCES: Calculations using National Student Clearinghouse, Delgado Community College, Louisiana 
Technical College, University of New Mexico, Borough of Manhattan Community College, Hostos Community 
College, and Ohio Board of Regents transcript data. 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates for all sites, excluding California, are adjusted by research cohort and campus. For California, 
estimates are adjusted by research cohort and workshop region. 

Only the first two cohorts are shown for Opening Doors Louisiana. 

a Represents data pulled from the National Student Clearinghouse. Data were not found for 305 students (6.6%) 
of the sample. 

b Due to the timing of scholarship-eligible terms and data limitations, some of these measures are proxies. For 
Louisiana, this represents the proportion of students who earned 6 or more credits with a "C" or better overall in the 
first term of their second year (depending on the cohort, this could be a fall, spring or summer term). Program 
group students were no longer eligible for a performance-based scholarship during this time. For New Mexico, this 
represents the proportion of students who earned 15 or more credits with a "C" or better overall in their first term of 
their second year (a fall term), and includes students who earned the award over the winter intersession. Program 
group students were still eligible for a performance-based scholarship during this time. For New York, this 
represents the proportion of students who earned 6 or more credits with a "C" or better in their first term of their 
second year (depending on the cohort, this could be a fall or spring term). Program group students were no longer 
eligible for a performance-based scholarship during this time. For Ohio, this represents the proportion of students 
who earned 12 or more credits in the first term of their second year (depending on the cohort, this could be a fall, 
winter, or spring term). Program group students were no longer eligible for a performance-based scholarship during 
this time. 

c Due to the timing of scholarship-eligible terms and data limitations, some of these measures are proxies. For 
Louisiana, this represents the proportion of students who earned 6 or more credits with a "C" or better overall in 
their second term of their second year (depending on the cohort, this could be a fall, spring or summer term). 
Program group students were no longer eligible for a performance-based scholarship during this time. For New 
Mexico, this represents the proportion of students who earned 15 or more credits with a "C" or better overall in 
their second term of their second year (a spring term), and does not include students who earned the award over the 
summer. Program group students were still eligible for a performance-based scholarship during this time. For New 
York, this represents the proportion of students who earned 6 or more credits with a "C" or better in their second 
term of their second year (depending on the cohort, this could be a fall or spring term). Program group students 
were no longer eligible for a performance-based scholarship during this time. For Ohio, this represents the 
proportion of students who earned 12 or more credits in their second term of their second year (depending on 
cohort, this could be a fall, winter, or spring term). Program group students were no longer eligible for a 
performance-based scholarship during this time. 

d Due to the timing of scholarship-eligible terms, this measure is calculated slightly differently for students at 
Sinclair Community College. For students in the fall 2008 cohort, this represents registration in the spring 2009 
quarter. For students in the winter 2009 cohort, this represents registration in the summer 2009 quarter (over which 
students were not eligible for a scholarship). For students in the spring 2009 cohort, this represents registration in 
the fall 2009 quarter. For students at Lorain County and Owens Community Colleges, depending on the cohort, this 
represents the fall or spring term. Program group students were no longer eligible for a performance-based 
scholarship during this time. 



The Performance-Based Scholarship Demonstration 
Table 5 


Ohio Degree Attainment During the First Three Years 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

Standard 

Error 

Earned any degree as of end of: 

First year 

9.5 

8.5 

1.1 

1.2 

Second year 

20.5 

16.9 

3.6 ** 

1.7 

Third year 

26.9 

23.3 

3.5 * 

1.8 

Highest degree earned as of end of third year 3 : 

Certificate 

6.5 

6.0 

0.5 

1.0 

Associate's 

19.2 

16.5 

2.7 * 

1.6 

Bachelor's 

1.2 

0.9 

0.3 

0.4 

Sample size (total = 2,285) 

1,359 

926 




SOURCES: Calculations using Ohio Board of Regents degree data. 

NOTES: A two-tailed t-test was applied to differences between the research groups. Statistical significance levels 
are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by research cohort and campus. 

Estimates are based on most recent data available and may change with additional rounds of processing. 
"Categories listed are mutually exclusive. 







The Performance-Based Scholarship Demonstration 
Table 6 

Year One Financial Aid Outcomes 


Program 

Sample 

Size 

Program 

Group 

Control 

Group Difference 


Percent 

Change 

Total financial aid received ($) 

New Mexico 

1,081 

11,098 

10,167 

932 

*** 

9.2 

New York 

1,502 

8,048 

5,839 

2,209 

*** 

37.8 

Ohio 

2,285 

7,947 

7,445 

502 

*** 

6.7 

Total performance-based scholarship received ($) 

New Mexico 

1,081 

1,440 

0 

1,440 

*** 

- 

New York 

1,502 

2,050 

0 

2,050 

*** 

- 

Ohio 

2,285 

765 

-1 

766 

*** 

- 

Total loans received ($) 

New Mexico 

1,081 

1,179 

1,525 

-347 

** 

-22.7 

New York 

1,502 

1,042 

1,033 

10 


0.9 

Ohio 

2,285 

2,853 

3,187 

-334 

*** 

-10.5 


SOURCES: Calculations using CUNY Institutional Research Database, University of New Mexico, Lorain 
County Community College, Owens Community College, and Sinclar Community College financial aid data. 

NOTES: A two-tailed t-test was applied to differences between the research groups. Statistical significance levels 
are indicated as *** = 1 percent, ** = 5 percent, and * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by research cohort and campus. 

The financial aid outcomes for the New York study represent financial aid awarded, which is distinct from 
financial aid received. The financial aid outcomes for the New Mexico and Ohio studies represent financial aid 
received. 






The Performance-Based Scholarship Demonstration 
Table 7 

Credits Earned as of End of First Year at Five Sites, by Subgroup 


Subgroup 

Sample 

Size 

Program 

Group 

Control 

Group Difference 

Standard 

Error 

Difference 

Between 

Subgroups 

Gender 

Male 

2,588 

17.5 

15.7 

j g *** 

0.4 

t 

Female 

4,383 

17.2 

16.2 

j o *** 

0.3 


Sample size 

6,971 






HisDanic 

Yes 

2,845 

17.9 

16.8 

1 j *** 

0.4 


No 

4,033 

16.9 

15.4 

1 5 *** 

0.3 


Sample size 

6,878 






Parent 

Yes 

3,721 

15.5 

14.1 

2 4 *** 

0.3 


No 

3,212 

19.4 

18.3 

j 1 *** 

0.3 


Sample size 

6,933 






Younger than 20 years old 

Yes 

1,826 

21.6 

20.6 

1.0 ** 

0.5 


No 

5,145 

15.8 

14.4 

2 4 *** 

0.3 


Sample size 

6,971 






First in family to attend college 

Yes 

2,211 

16.9 

15.3 

1 ^ *** 

0.4 


No 

4,574 

17.5 

16.5 

j o *** 

0.3 


Sample size 

6,785 






Employed at beginning of program 

Yes 

3,424 

17.2 

15.9 

1 3 *** 

0.3 


No 

3,413 

17.5 

16.2 

1 3 *** 

0.4 


Sample size 

6,837 






Program with services component 3 

Yes 

3,184 

19.0 

17.7 

1 2 *** 

0.4 


No 

3,787 

15.9 

14.6 

j 4 *** 

0.3 


Sample size 

6,971 







(continued) 



















Table 7 (continued) 


SOURCE: Calculations using Baseline Information Form (B1F) data and Pima Community College, Flillsborough 
Community College, Lorain County Community College, Owens Community College, Sinclair Community 
College, University of New Mexico, Borough of Manhattan Community College, and Hostos Community 
College transcript data. 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels are 
indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by research cohort and campus. 

a The programs with a services component are located in Arizona, Florida, and New Mexico. 



The Performance-Based Scholarship Demonstration 
Table 8 

Registration at Beginning of Second Year at Six Sites, by Subgroup 


Difference 

Sample Program Control Standard Between 


Subgroup 

Size 

Group 

Group Difference 

Error Subgroups 

Gender 

Male 

3,956 

71.3 

68.5 

2.8 * 

1.5 

Female 

6,903 

72.0 

71.2 

0.8 

1.1 

Sample size 

10,859 





HisDanic 

Yes 

5,170 

74.3 

71.7 

2.6 ** 

1.3 

No 

5,562 

69.4 

68.9 

0.5 

1.3 

Sample size 

10,732 





Parent 2 

Yes 

3,484 

61.4 

60.8 

0.6 

1.7 

No 

2,701 

68.6 

68.0 

0.5 

1.8 

Sample size 

6,185 





Younger than 20 years old 

Yes 

6,173 

79.3 

77.5 

1.9 * 

1.1 

No 

4,687 

61.7 

60.8 

0.9 

1.4 

Sample size 

10,860 





First in family to attend college 

Yes 

4,442 

72.8 

70.5 

2.3 * 

1.4 

No 

6,215 

71.0 

70.3 

0.8 

1.2 

Sample size 

10,657 





Employed at beginning of program 3 

Yes 

3,100 

65.8 

65.5 

0.4 

1.7 

No 

3,003 

63.7 

62.3 

1.4 

1.8 

Sample size 

6,103 





Program with services component 1 ’ 

Yes 

2,431 

67.7 

67.7 


1.9 

No 


72.9 

71.0 

1.9 * 

1.0 

Sample size 

10,860 






(continued) 
















Table 8 (continued) 


SOURCE: Calculations using Baseline Information Form (B1F) data, National Student Clearinghouse data, and 
Pima Community College, Flillsborough Community College, Lorain County Community College, Owens 
Community College, Sinclair Community College, University ofNew Mexico, Borough of Manhattan 
Community College, and Flostos Community College transcript data. 

NOTES: A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

A two-tailed t-test was applied to differences of impacts between subgroups. Statistical significance levels are 
indicated as: fff = 1 percent; f f = 5 percent; f = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates for all sites, excluding California, are adjusted by research cohort and campus. For California, 
estimates are adjusted by research cohort and workshop region. 

Results shown include the first two of three cohorts in Arizona and Florida. 

a Data on parental status and employment were not collected for California students. 

b The programs with a services component are located in Arizona, Florida, and New Mexico. 



